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PROMOTING  LEARNER  REFLECTION: 

ISSUES  AND  DIFFICULTIES  EMERGING  FROM  A  THREE- YEAR  STUDY 


Background 

Although  it  is  generally  believed  that  reflection  is  a  powerful  learning  tool,  recent 
experiences  of  these  authors  with  three  different  groups  of  military  cadets  suggest  that 
deep  reflection  is  difficult  to  promote.  Reflection  as  defined  here  means  to  think  for  an 
extended  time  about  a  set  of  recent  experiences  looking  for  commonalties,  differences, 
and  interrelations  beyond  their  superficial  elements  (Schon,  1987;  van  Wright,  1992; 
Yinger  &  Clark,  1981).  Such  reflection  would  also  promote  linking  these  new 
experiences  to  earlier  ones  and  also  to  what  some  authors  (e.g.,  Anderson,  1984;  Berliner, 
1986)  have  referred  to  as  the  individual's  mental  schemata.  Thus,  from  the  instructor's 
point  of  view,  promoting  reflection  should  result  in  learners  thinking  more  deeply  about  a 
topic,  thereby  developing  a  more  complex  and  interrelated  mental  schema,  and 
facilitating  recall  of  important  concepts,  principles,  and  rules  (Brown,  Bransford,  Ferrear, 
&  Campione,  1983)  Presumably  an  additional  benefit  of  reflection  would  be  assisting  the 
learner  in  developing  higher  order  thinking  skills  (Sparks-Langer,  &  Colton,  1991) 

However,  as  mentioned,  promoting  reflection  among  military  cadets  by  means  of 
written  responses  to  required  "diaries"  has  proven  quite  difficult.  The  diaries  were 
prepared  by  the  authors  to  accompany  a  new  and  innovative  field  experience  course  for 
future  civil  engineers.  The  underlying  concept  of  the  course  is  to  provide  cadets  with  a 
set  of  real  world  experiences  before  they  study  the  formal  theories  and  principles  of  civil 
engineering.  For  example,  cadets  construct  actual  wood  frame  buildings,  experiment 
with  water  flow  in  open  channels,  erect  steel  structures  such  as  bridges  and  water  towers, 
and  design,  construct,  and  test  concrete  beams  during  a  summer-long  field  experience.  In 
the  fall  they  then  begin  classes  that  cover  these  topics  employing  more  conventional 
instructional  strategies. 

The  structure  of  this  summer  experience  is  grounded  in  the  concepts  of  situated 
cognition  (Choi  &  Hannafin,  1995;  Brown,  Collins,  &  Duguid,  1989)  and  authentic  tasks 
(Brown,  et  al.,  1989;  Newmann,  1991).  Situated  cognition  assumes  a  "real-world" 
environment  in  which  the  learner  thinks  about  and  acts  on  real  problems.  Knowledge  is 
assumed  to  result  as  a  by-product  of  this  engagement  and  transfer  to  similar,  but  not 
identical  situations  is  believed  to  be  enhanced  (Brown  &  Duguid,  1993).  Authenticity  is 
assumed  to  have  motivational  benefits  as  well  as  facilitate  the  learner's  ability  to  judge 
the  appropriateness  of  any  specific  knowledge,  technique,  or  process  when  faced  with 
another  set  of  conditions. 

Since  a  primary  goal  of  the  summer  program  is  to  assist  cadets  in  developing  a 
cognitive  schema  for  connecting  the  many  engineering  principles,  practices,  and  theories 
they  subsequently  study  in  courses,  it  was  decided  to  ask  them  to  respond  to  assigned 
diary  items  on  a  regular  schedule  during  the  course.  The  intent  was  to  provide  cadets 
with  a  set  of  questions  that  would  require  reflection  as  they  were  preparing  their 
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individual  responses.  For  example,  cadets  who  had  declared  a  major  in  environmental 
engineering  were  asked  to  explain  what  would  be  valuable  to  cadets  majoring  in  other 
branches  of  engineering  from  a  just  completed  field  trip  to  a  wastewater  treatment  plant. 
The  assumption  was  that  they  would  reflect  on  the  environmental  elements  of  what  they 
had  learned  and  that  this  reflection  would  be  generally  useful  to  other  engineering 
specialties;  thereby  enhancing  their  own  connection  of  key  ideas  and  infonnation.  A 
parallel  question  asked  cadets  majoring  in  civil  engineering  what  they  thought  was 
valuable  for  themselves  from  the  just  completed  field  trip  even  though  its  primary  focus 
was  on  environmental  topics.  For  example,  civil  engineers  have  to  design  and  construct 
the  physical  plants  associated  with  wastewater  treatment. 

Review  of  the  cadets’  responses  to  these  and  other  questions  indicated  very  little 
reflection  had  occurred  and  infonnal  interviews  with  cadets  confirmed  this  opinion.  With 
but  a  few  notable  exceptions,  cadets  across  three  different  years  generally  did  not  produce 
responses  indicating  any  deep  reflection.  This  despite  systematic  attempts  by  the  authors 
to  experiment  with  different  question  formats,  to  explain  in  person  and  in  writing  what 
was  the  purpose  of  the  questions,  and  to  describe  to  cadets  the  value  of  reflection  to 
gaining  maximum  benefit  from  the  summer  program.  Although  these  results  were 
disappointing,  they  are  consistent  with  the  research  literature  on  promoting  reflection  that 
generally  indicates  it  is  difficult  to  accomplish  (Stamper,  1996). 

This  consistent  lack  of  success  in  promoting  reflection  has  caused  the  authors  to 
themselves  reflect  on  what  might  be  the  reasons  for  the  cadets’  behavior.  (Our  reflection 
reaffirmed  our  belief  that  reflection  is  a  useful  behavior!)  Quickly  ruled  out  was  the 
cadet's  basic  intelligence.  These  individuals  are  a  highly  select  pool  from  which  history 
indicates  many  highly  qualified  future  military  leaders  will  emerge.  Also,  since  they 
were  rising  Juniors,  their  base  level  of  motivation  to  succeed  was  very  high  or  they 
would  not  have  succeeded  in  their  first  two  years  at  the  Air  Force  Academy.  Thus, 
intelligence  and  baseline  academic  motivation  could  be  ruled  out  as  explanations  of  their 
poor  reflection.  It  is  important  to  note  however,  that  we  assume  the  cadets  view  the 
summer  program  as  an  academic  activity.  If  the  cadets  did  not  view  the  summer  program 
as  an  academic  activity,  then  the  dynamics  and  motivation  to  reflect  would  be  decidedly 
different  (and  most  likely  altogether  absent).  Similarly,  their  academic  achievement 
during  their  first  two  years  provides  clear  evidence  of  their  ability  to  achieve  in  a  military 
environment  (albeit  an  artificial  one). 

Review  of  the  relatively  limited  empirical  research  literature  on  reflection  (e.g., 
Sparks-Langer  &  Colton,  1991;  Stamper,  1996)  and  our  own  reflection  has  led  to 
identifying  ten  variables  we  believe  independently  and  interactively  affected  cadets'  lack 
of  reflective  behavior.  We  further  believe  many  of  these  variables  are  likely  to  be 
operational  in  other  fonnal  learning  settings  with  other  learners  and  provide  fruitful  areas 
for  future  research.  Parenthetically,  we  note  that  there  is  a  substantial  body  of  literature 
(e.g.,  Clift,  Houston,  &  Pugach,  1990;  Ross,  1989),  expressing  the  opinion  that  reflection 
is  beneficial  without  providing  supporting  research  data.  Nonetheless,  even  lacking  solid 
research  evidence  to  support  their  decision  making,  instructional  designers  can  likely 
enhance  the  effectiveness  of  instruction  by  considering  these  variables  as  they  plan 
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instruction  that  has  as  one  of  its  intents  promoting  reflection.  The  variables  can  be 
grouped  into  three  main  categories:  Learner  Characteristics,  Environmental 
Characteristics,  and  Reflection  Task  Characteristics. 


Variables  affecting  the  reflection  process 

Learner  Characteristics 

•  Learner's  skill  and  experience  in  reflective  thinking 

•  Breath  of  learner's  knowledge  of  the  content  area 

•  Learner's  motivation  to  complete  the  reflection  task 

•  Mental  preparation  (mental  set)  of  the  learner  for  the  reflection  task 

•  Degree  of  security  felt  by  learner  in  reporting  actual  reflections  versus  perceived 
desired  responses 

Environmental  Characteristics 

•  Nature  of  the  physical  environment  in  which  reflection  is  being  expected 

•  Nature  of  the  interpersonal  environment  in  which  reflection  is  to  occur 

Reflection  Task  Characteristics 

•  Nature  of  the  stimulus  questions,  directions,  or  probes 

•  Format  required  for  the  learner  to  report  on  results  of  reflecting 

•  Quality  of  the  feedback  provided  the  learner  following  reflection 

•  Consequences  of  Reflection 

Each  of  these  variables  will  now  be  considered  separately,  but  obviously  their 
degree  of  influence  in  any  particular  setting  and  their  interactive  effects  might  also  be 
considered. 


Learner  Characteristics 

Learner's  skill  and  experience  in  reflective  thinking 

The  ability  to  reflect  is  a  learned  behavior  that  can  be  cultivated  by  the  individual 
overtime.  (Sparks-Langer  &  Colton,  1991)  Although  how  reflective  an  individual  may 
be  to  some  degree  a  personality  trait,  skill  in  reflecting  can  be  developed  by 
systematically  designing  appropriate  learning  experiences  for  students.  Von  Wright 
(1992)  makes  an  important  distinction  between  reflection  as  a  specific  skill  that  is 
situated  in  a  specific  situation  and  reflection  as  a  general  capacity  that  can  be  applied 
across  varied  settings.  Ross  (1989)  has  cautioned,  however,  that  it  is  relatively  difficult 
to  teach  reflection  as  a  generalizable  skill.  Learning  experiences  that  might  promote  one 
or  the  other  of  these  levels  of  reflection  skills  include  describing  the  desired  behavior, 
providing  case  examples,  modeling  the  behavior,  providing  a  series  of  graded  practice 
exercises  and  accompanying  guidance,  and  providing  feedback  on  performance.  These 
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suggested  instructional  strategies  for  enhancing  skills  in  reflecting  are  based  on  our  own 
experience  as  instructional  designers  and  are  deserving  of  research  investigation  since 
little  empirical  literature  presently  exists  for  the  topic.  Nonetheless,  we  believe  that  the 
cadets  possessed  different  levels  of  competence  in  being  able  to  reflect  and  that  this 
variance  impacted  their  ability  to  engage  in  structured  reflection  activities. 

Breadth  of  learner's  knowledge  of  the  content  area 

The  ability  to  reflect  on  a  specific  topic  is  related  to  how  much  one  already  knows 
about  that  topic.  If  one’s  schema  for  a  topic  is  limited,  there  is  less  ability  to  relate  new 
information  to  it.  This  is  true  for  any  form  of  cognitive  learning  from  recall  of  specific 
factual  infonnation  to  application  of  principles  or  rules  (Schon,  1987).  Thus,  it  is  not 
surprising  that  reflection  is  similarly  affected  by  the  amount  of  prior  knowledge.  In  the 
cadet  diaries  an  item  related  to  wood  frame  building  construction  practices  was  answered 
in  more  thoughtful  detail  by  those  who  had  prior  construction  experience  than  those  who 
had  never  before  lifted  a  hammer.  Those  who  had  worked  around  construction  sites  had 
a  better  grasp  of  work  flow  and  could  describe  in  more  detail  the  effects  on  task 
completion  and  safety  of  how  the  job  site  was  organized  than  those  without  such  prior 
experience.  The  precise  nature  of  how  prior  knowledge  and  experience  impacts  one's 
ability  to  reflect  deserves  more  research.  Work  on  mental  models  and  how  to  measure  an 
individual’s  existing  schema  is  being  conducted  by  a  number  of  researchers  including 
Kyllonen  and  Shute  (1989)  and  Sasse  (1991). 

Learner's  motivation  to  complete  the  reflection  task 

Both  internal  and  external  sources  of  motivation  may  affect  the  amount  and 
quality  of  reflection  engaged  in  by  the  learner.  While  it  may  be  possible  to  promote  deep 
reflection  by  manipulation  of  external  factors  (e.g.,  grades,  threats,  rewards,  etc.),  none  of 
these  external  contingencies  was  used  with  the  cadets  and  we  would  hypothesize  that 
they  would  not  have  been  successful.  Although  subject  to  experimental  verification, 
intuition  tells  us  that  overt  contingency  management  strategies  will  produce  superficial 
reflection  and  reporting.  It  may  be  that  some  subtle  forms  of  social  reinforcement  would 
be  successful,  but  if  so,  these  specific  forms  are  likely  to  be  highly  individualized. 

Internal  motivation  by  its  nature  is  difficult  to  elevate  and  even  more  difficult  to 
accurately  estimate  or  measure.  Our  attempts  to  raise  internal  motivation  by  pointing  out 
the  value  of  reflection  did  not  appear  to  have  any  impact.  This  was  the  case  whether  the 
general  benefits  were  described  (e.g.,  "the  benefits  of  reflecting  to  learning  are  generally 
recognized"),  or  specific  reference  was  made  to  the  benefits  (e.g.,  "to  assist  you  in 
remembering  the  general  flow  of  steps  in  framing  a  building  ....")  in  individual  diary 
items.  It  may  be  that  other  strategies  such  as  creating  a  mental  challenge,  organizing  the 
learners  into  pairs  to  create  a  social  learning  environment,  or  forming  competitive  teams 
would  enhance  motivation,  but  again  the  effectiveness  of  these  and  other  strategies  for 
promoting  reflection  awaits  verification.  Under  some  conditions  it  seems  likely  that 
feedback  during  and  after  reflection  might  enhance  motivation.  External  consequences  of 
not  engaging  fully  with  the  task  may  also  have  some  impact  on  the  motivation  to  reflect. 
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However,  as  will  be  discussed  later,  we  believe  creating  highly  artificial  external  rewards 
or  punishments  may  actually  reduce  the  level  of  reflection  although  more  extensive 
"work  products"  might  be  generated  to  satisfy  those  in  authority. 

Mental  preparation  (mental  set)  of  the  learner  for  the  reflection  task 

Although  the  mental  set  of  the  individual  might  be  considered  as  a  motivational 
variable,  it  is  described  separately  here  to  highlight  its  probable  importance  to  promoting 
reflection.  Observation  of  the  cadets  indicated  that  in  many  instances  they  were  not 
mentally  prepared  to  engage  in  deep  reflection  after  completing  their  daily  work 
assignments.  The  tasks  they  had  been  engaged  with  were  for  the  most  part  physical  such 
as  pounding  nails,  placing  and  finishing  concrete,  and  erecting  steel  structures.  Even 
when  the  tasks  had  significant  cognitive  components,  such  as  surveying  for  a  new  road, 
cadets  did  not  appear  ready  to  engage  in  thoughtful  reflection  at  the  end  of  a  day  of  hard 
physical  labor.  Although  we  were  not  able  to  interview  them  at  this  point  about  their 
readiness  to  reflect,  informal  conversations  lead  to  the  conclusion  that  they  wanted  to 
think  and  talk  about  the  experiences  at  a  much  "simpler"  level.  Recalling  humorous  (and 
perhaps  serious)  events  of  the  day,  kidding  each  other  about  their  skill  (or  lack  thereof), 
and  telling  anyone  who  would  listen  how  hard  they  had  been  working,  dominated  their 
immediate  conversation.  When  these  topics  were  exhausted  the  conversation  often 
turned  to  food,  getting  cleaned  up,  and  what  they  planned  to  do  after  dinner;  further 
indicating  a  lack  of  the  mental  set  to  think  deeply  about  the  day's  activities  and  relate 
them  to  each  other  and  to  prior  experiences. 

Although  there  is  some  literature  describing  strategies  for  establishing  a  mental 
set  (e.g.,  Gagne,  Briggs,  &  Wager,  1992),  additional  research  on  assessment  of  mental  set 
as  well  as  on  other  strategies,  especially  for  use  in  the  kinds  of  settings  we  have 
described,  would  be  desirable.  Timing  also  is  related  to  the  ability  to  establish  a  mental 
set.  It  is  doubtful  that  the  cadets  we  observed  would  have  been  receptive  to  any  strategy 
designed  to  establish  the  mental  set  necessary  for  reflection  immediately  after  leaving 
their  work  sites.  Despite  having  been  working  hard,  often  in  the  hot  sun,  they  were  eager 
to  talk;  but  what  they  wanted  to  talk  about  was  quite  predicable  and  the  conversations 
were  usually  very  animated.  It  seemed  important  that  they  have  the  opportunity  for  this 
type  of  social  interaction  with  peers  and  staff,  and  any  attempts  to  steer  them  to  a  deeper 
level  of  thinking  were  typically  doomed.  Thus,  timing  of  any  attempt  to  promote  an 
appropriate  mental  set  must  also  be  considered. 

Experimentation  with  "pre-reflection"  strategies  to  set  the  stage  for  what  will  be  expected 
following  the  instructional  event  should  be  explored  as  well  as  specific  strategies 
immediately,  and  perhaps  well  after,  the  event  to  examine  their  impact  on  reflection. 
Group  reflection  may  also  hold  possibilities  which  were  never  tested  in  the  current 
situation. 

Degree  of  security  felt  by  learner  in  reporting  actual  reflections  versus  perceived  desired 
responses 
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Candor  and  completeness  of  responses  to  diary  items  varies  depending  on  how 
respondents  view  those  who  will  be  reviewing  their  submissions.  When  there  is 
confidence  in  the  professionalism  and  integrity  of  reviewers,  the  amount  and  quality  of 
responses  will  be  enhanced.  This  is  particularly  true  when  items  call  for  making 
judgments  about  the  worth  of  an  activity  or  the  quality  of  the  instruction.  This  type  of 
reflection  can  be  used  to  promote  thinking  about:  what  was  and  was  not  included  that  the 
learner  wanted  or  needed  to  learn;  what  the  designer  of  the  instruction  may  have 
incorrectly  assumed  about  the  learner’s  entering  knowledge  or  skill;  or  why  the 
instruction  was  or  was  not  effective.  Similarly,  items  might  ask  for  suggestions  on  what 
might  be  improved  about  the  program  or  request  specific  examples,  scenarios  or 
illustrations  for  inclusion  in  future  versions  of  the  instruction.  These  types  of  items  may 
cause  learners  to  reflect  on  the  content  as  well  as  the  instructional  strategies  thereby 
promoting  deeper  understanding. 

However,  unless  the  learners  feel  absolutely  confident  they  can  tell  the  truth 
without  suffering  negative  consequences  they  are  unlikely  to  provide  honest  answers. 

Not  only  will  the  answers  be  suspect,  but  it  is  also  likely  they  will  not  even  think 
seriously  about  the  item  itself  before  formulating  what  they  believe  will  be  a  "safe" 
response.  Playing  it  safe  will  supersede  reflection  under  almost  any  conditions. 

Environmental  Characteristics 

Nature  of  the  physical  environment  in  which  reflection  is  being  expected 

The  opportunity  for  the  learner  to  establish  an  appropriate  mental  set  for  reflecting 
is  also  related  to  the  nature  of  the  physical  environment  in  which  reflection  is  expected  to 
take  place.  Cadets  often  were  asked  to  complete  diaries  at  the  end  of  long  and  physically 
demanding  days  in  the  field.  However,  even  after  they  had  showered  and  eaten  they  had 
few  physical  amenities  available  since  at  least  part  of  the  time  they  were  living  under 
less-than-optimal  learning  conditions.  Living  quarters  were  tight,  tents  were  lit  with  bare 
light  bulbs,  and  privacy  was  a  scarce  commodity.  Tables  or  desks  were  generally  not 
available,  often  forcing  cadets  to  complete  the  diaries  using  clipboards  or  notebooks  as 
writing  surfaces.  Given  that  both  the  stimulus  to  think  (the  diary  item)  and  that  the 
recording  of  the  results  of  their  thinking  had  to  be  handwritten  under  marginally 
supportive  physical  conditions,  it  was  obvious  that  the  physical  enviromnent  had  a 
significant  negative  influence  on  any  attempts  at  reflective  behavior.  Actually,  our 
results,  especially  the  last  year,  may  have  been  the  result  of  an  interaction  of  a  number  of 
key  variables,  which  alone  may  not  have  impacted  reflection  in  a  negative  way,  but  taken 
together,  resulted  in  dramatically  poor  reflection.  Although  one  could  argue  that 
reflection  is  a  mental  task,  the  poor  physical  surroundings  did  have  a  negative  impact  on 
what  was  recorded  in  the  diaries  and  had  a  similar  negative  effect  on  the  mental  activity 
that  preceded  it.  There  was  no  evidence  that  cadets  had  reflected  but  simply  not  recorded 
the  results  of  that  reflection.  It  seems  safe  to  conclude  that  little  reflection  of  the  type 
desired  took  place. 
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Other  factors  that  may  contribute  to  a  poor  physical  environment,  even  under  less 
primitive  conditions,  include:  competing  stimuli  (e.g.,  TV,  personal  conversations,  etc.), 
ambient  noise,  poor  ventilation,  high  or  low  temperature,  uncomfortable  furniture,  and 
perhaps  even  the  color  of  the  surroundings.  The  question  of  how  extreme  any  of  these 
conditions  must  be  (alone  or  in  concert  with  other  variables)  before  they  begin  to  impact 
one's  ability  to  reflect  is  another  area  of  potential  research. 

Nature  of  the  interpersonal  environment  in  which  reflection  is  to  occur 

Although  not  explicitly  examined  during  our  three  years  of  working  with  the 
cadets,  the  nature  of  the  interpersonal  environment  probably  affects  the  amount  and 
quality  of  reflection  that  occurs.  Due  to  the  highly  competitive  conditions  at  the 
academy,  collaborative  learning  is  not  a  routine  experience.  In  fact,  the  diary 
assignments  were  seen  as  an  individual  requirement  that  each  cadet  was  expected  to 
complete  and  turn  in  for  review.  Also,  it  is  an  Air  Force  Academy  regulation  that  if  a 
cadet  receives  any  assistance  of  any  type  (no  matter  how  small)  on  any  assignment,  this 
must  be  noted  on  the  work  that  is  submitted.  Failure  to  adhere  to  this  requirement  has 
serious  consequences.  Thus,  no  interaction  among  cadets  would  occur  during  diary 
completion  unless  they  were  specifically  instructed  to  do  so. 

It  may  be  that  environments  that  require  (or  at  least  permit  and  promote) 
interpersonal  interaction  will  result  in  greater  reflection.  This  hypothesis  is  based  on 
social  learning  theory.  Social  interaction  may  enhance  motivation  and  prolong 
engagement  with  the  task.  Social  interaction  would  almost  certainly  bring  forth  more 
information  and  ideas  that  could  be  shared  by  those  involved  and  perhaps  result  in 
thinking  more  deeply  about  the  topic.  Exhaustive  review  of  cadet  comments  from 
individual  diary  items  suggests  that  this  sharing  of  information  would  have  been 
beneficial  since  they  often  reported  different  ideas  and  insights  from  the  same  experience. 
Interaction  with  other  cadets,  perhaps  under  the  guidance  of  an  instructor,  staff  member, 
or  more  senior  cadet  skilled  in  leading  group  discussion,  might  have  promoted  reflection. 
However,  Stamper  (1996)  found  that  peer  dialogue  in  a  computer  mediated  environment 
was  not  well  accepted  by  her  subjects  and  generally  was  not  successful.  However,  here  is 
a  place  where  intervention  during  the  pre -training  for  the  flight  commanders  may  have  a 
payoff.  The  payoff  is  in  terms  of  teaching  them  how  to  be  better  reflective  stimuli  and 
probes  of  their  flight  membership  to  help  facilitate  the  reflection  process. 

Reflection  Task  Characteristics 

Nature  of  the  stimulus  questions,  directions,  or  probes 

The  nature  of  the  stimulus  to  reflect  also  will  impact  the  quality  of  the  reflection. 
Surbeck,  Han,  and  Moyer  (1991)  identify  three  different  levels  of  reflection  that  they 
label  reaction,  elaboration,  and  contemplation.  They  believe  the  nature  of  the  stimulus  or 
directions  initially  provided  the  learner  as  well  as  the  feedback  provided  to  them  after 
initial  reflection,  will  determine  the  extent  to  which  they  reach  the  contemplation  level  of 
reflection. 
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Over  three  years  of  experimenting  with  different  forms  of  questions  and  probes 
we  have  determined  that  most  of  the  typical  forms  of  questions  found  on  tests  and 
quizzes  do  not  promote  serious  reflection  even  when  the  students  are  specifically 
instructed  to  do  so  and  the  benefits  of  reflection  to  enhancing  their  learning  are  described. 
Obviously  "fact"  questions  (e.g.,  What  are  the  functional  areas  of  an  air  base?)  do  not 
promote  reflection.  But  posing  hypothetical  situations  (e.g.,  Assume  you  have  inherited  a 
significant  sum  of  money  and  wish  to  buy  land  in  an  environmentally  sensitive  area  on 
which  to  build,  what  factors  will  go  into  your  decision  and  why?)  produced  similarly 
disappointing  results.  In  contrast,  the  most  successful  probe  asked  learners  to  write  a  one 
page  letter  to  a  parent,  sibling,  or  other  significant  person  in  their  lives.  However  their  is 
a  limit  to  the  number  of  this  specific  type  of  question  there  can  be  the  pool  of  items,  so 
the  problem  remains  how  to  develop  other  questions  or  probes  to  promote  reflection? 

Almost  invariably  during  the  first  two  summers,  cadets  used  vivid  language, 
provided  rich  detail,  and  reported  on  the  value  (or  lack  there  of)  of  the  past  two  to  three 
weeks  of  intensive  activity.  It  appears  that  stimuli  containing  an  evaluative  dimension 
seem  to  promote  more  recall  and  reflection  than  those  that  require  dealing  only  with 
recalling  and  describing  the  substantive  content.  Casting  diary  items  in  the  form  of 
personal  correspondence  to  a  person  with  whom  they  are  close  also  played  a  role  as  well, 
since  many  responses  included  personal  comments  for  this  type  of  item.  For  example, 
comments  such  as,  “How  is  my  dog  doing?”  suggest  that  the  letter  writing  item  took  on  a 
strong  personal  reality  even  though  the  diary  was  turned  in  and  not  actually  sent  home. 
Even  letters  to  a  future  cadet  in  the  program  often  contained  personal  comments  such  as, 
"Have  a  good  time"  and  "You  will  really  enjoy  the  beach  time." 

Based  on  a  small  sample  case  study,  Stamper  (1996)  concluded  that  explicit 
guidance  promoted  reflection  for  only  one-half  of  her  subjects,  but  why  this  was  the  case 
is  not  known.  As  of  now  there  is  little  in  the  way  of  specific  recommendations  that  can 
be  made  to  instructional  designers  as  to  how  much  of  what  type  of  guidance  to  provide 
learners  when  attempting  to  promote  reflection. 

Format  required  for  the  learner  to  report  on  results  of  reflection 

Another  variable  not  examined  in  our  work,  but  that  may  affect  both  the  reflection 
on,  and  reporting  of,  the  results,  is  the  format  in  which  they  are  reported.  Yinger  and 
Clark  (1981)  believe  that  the  requirement  that  the  results  of  reflection  be  written  down  is 
more  powerful  than  reporting  them  orally.  However,  handwriting  is  slow,  requires  a 
writing  surface,  and  revisions  or  extensions  of  what  has  been  recorded  are  less  likely  than 
for  products  produced  on  a  word  processor.  Word  processing  has  the  advantage  of  easy 
revision,  but  requires  that  equipment  be  readily  available.  Both  handwritten  and  typed 
responses  do  provide  a  permanent  record,  a  disadvantage  of  verbal  reports  (unless  they 
are  recorded).  For  our  research  purposes,  all  cadet  diary  entries  for  the  first  two  years 
were  transcribed  so  they  could  be  analyzed  using  qualitative  software,  but  that  involves 
considerable  time  and  expense.  Verbal  reports  might  be  more  extensive  since  most 
individuals  can  talk  faster  than  they  can  write  or  type  and  may  feel  more  obligated  when 
preparing  to  make  a  verbal  presentation.  Verbal  reports  when  presented  "live"  could 


have  the  additional  advantage  of  interaction  of  the  learner  with  other  learners  and  with 
the  instructor  and  under  some  conditions  additional  "real  time"  reflection  might  occur. 
However,  the  demand  on  instructor  time  would  be  dramatically  increased  if  many 
learners  were  involved. 

In  summary,  we  believe  that  the  format  required  for  reporting  the  results  of 
reflection  can  influence  both  the  amount  of  reflection  that  takes  place  and  the  quantity 
and  quality  of  the  responses  generated.  Format  probably  interacts  with  a  number  of  other 
variables  and  any  format  decision  would  need  to  take  into  account  the  existing  skills  of 
the  learner  (e.g.,  typing,  handwriting,  verbal)  as  well  as  logistic  factors  (e.g.,  time, 
equipment).  Whether  a  permanent  record  is  desirable  or  required  is  another  instructional 
design  decision  when  selecting  a  format. 

Quality  of  the  feedback  provided  the  learner  following  reflection 

Feedback  can  take  many  forms  ranging  from  no  feedback,  to  acknowledging  that 
the  work  was  done,  to  commenting  on  how  well  it  was  done,  to  extending  beyond  or 
elaborating  on  what  was  submitted.  (Surbeck,  Han,  &  Moyer,  1991)  At  the  first  level, 
providing  no  feedback  had  a  substantial  negative  effect  on  cadet  behavior.  Cadets  did  not 
continue  to  submit  work  when  there  was  no  evidence  that  it  made  any  difference.  This 
was  demonstrated  during  the  third  year  of  the  program  when  cadets  were  given  little  if 
any  feedback  on  their  diary  submissions  due  to  a  change  in  course  management.  The 
predictable  result  was  that  the  quantity  and  quality  of  responses  diminished  substantially. 
Although  there  was  not  a  great  deal  of  reflection  evidenced  in  the  first  two  years,  at  least 
all  the  diaries  and  all  the  items  were  completed  whereas  in  the  third  year  many  diaries 
were  not  even  submitted  toward  the  end  of  the  program. 

Evaluative  feedback  can  take  many  fonns,  but  for  promoting  reflection  would 
require  more  than  just  assessing  whether  or  not  the  work  submitted  was  acceptable. 
Feedback  in  the  form  of  praise  from  a  respected  individual  such  as,  "You  did  a  nice  job 
on  that  item"  or  "I  like  the  analogy  you  used  in  your  answer"  would  likely  have  a  positive 
effect  on  motivation  to  think  more  deeply  before  responding  to  future  items. 

Extending  beyond  evaluative  feedback  might  have  even  more  powerful  effects. 
Providing  probes  such  as,  "Have  you  thought  about  how  a  skilled  operator  might  do 
this?"  or  "But  how  much  does  safety  really  get  compromised  when  you  don’t  use  safety 
shoes?"  may  cause  the  learner  to  continue  to  think  about  the  topic.  Pointing  out  other 
possibilities  such  as,  "Another  factor  you  might  consider  is  how  many  different  tools  will 
be  required  if  you  use  different  size  bolts  in  the  design,"  or  "But  what  if  the  rate  of  water 
flow  is  doubled?"  also  may  result  in  additional  thinking  about  relationships  among 
factors  not  previously  considered.  Although  such  enhanced  feedback  may  be  provided 
via  written  comments,  they  are  probably  most  powerful  when  used  interactively  in 
interpersonal  dialogue.  Carrying  on  a  dialogue  with  one  or  more  learners  about  the  work 
they  have  submitted  is  probably  the  ultimate  in  promoting  reflection  via  feedback.  But 
the  logistics  of  doing  so  and  having  discussion  leaders  who  are  skilled  in  the  content  and 
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possess  good  interpersonal  skills  may  be  beyond  the  capacity  of  the  system  to  provide; 
unless  it  is  computer  mediated  in  some  way. 

Clearly,  the  amount  and  quality  of  feedback  provided  affects  the  amount  and 
quality  of  reflection  taking  place.  Feedback  affects  future  motivation  to  reflect  and  when 
sufficiently  elaborated,  may  result  in  additional  reflection  on  the  original  topic.  As  in 
other  forms  of  learning,  feedback  is  an  important  variable  to  consider  when  designing 
instruction  to  promote  reflection.  However,  managing  an  extensive,  frequent, 
personalized,  and  elaborate  feedback  process  is  a  challenging  logistical  and  cognitive 
endeavor  when  a  large  number  of  learners  are  involved  as  any  classroom  teacher  will 
testify.  How  some  of  this  type  of  feedback  might  be  facilitated  within  a  computer 
mediated  communication  environment  provides  an  important  research  agenda  for  the 
future. 

Consequences  of  Reflecting 

Cadets  in  the  third  year  produced  much  less  "rich"  letters  even  though  the  same 
diary  items  were  employed.  The  letters  home,  requested  after  the  first  two  weeks  of  the 
program,  were  somewhat  less  thoughtful  than  in  previous  years,  but  generally  provided 
some  evidence  of  reflection.  However,  the  letters  to  a  future  cadet  in  the  program 
requested  at  the  end  of  the  program  were  generally  dismal  and  in  some  cases  a  blank 
sheet  of  paper  was  submitted.  The  substantial  reduction  in  feedback  was  largely  the 
result  of  a  significant  drop  in  the  amount  of  faculty  time  devoted  to  this  course  as  other 
department  priorities  emerged.  In  the  first  two  years,  extra  faculty  time  was  assigned  to  it 
since  in  was  still  in  the  developmental  stage.  However,  by  the  third  year  it  was  felt  the 
course  needed  to  operate  with  only  the  amount  of  faculty  time  required  of  other  courses. 
The  result  was  that  the  faculty  had  little  time  available  to  read  and  respond  to  the  diaries 
and  it  became  obvious  to  all  involved  that  there  would  be  no  negative  consequences  (and 
probably  no  positive  ones  either)  to  their  not  being  completed. 

Thus,  we  believe  the  poor  completion  of  the  diaries  was  a  direct  consequence  of 
the  staff  not  taking  the  diaries  seriously  and  when  it  became  known  to  the  cadets  that  not 
completing  the  diaries  had  no  consequences,  they  also  stopped  taking  them  seriously. 

This  is  a  totally  rational  behavior  on  the  part  of  cadets  and  indicates  how  the  variables 
being  described  interact  rather  than  operating  independently.  Despite  the  fact  that  some 
of  the  diary  items  were  of  proven  effectiveness  in  earlier  years,  they  did  not  work  well  in 
year  three  because  of  the  feedback  (albeit  indirect  and  unintentional)  that  the  diary 
assignments  were  not  important. 


SUMMARY 

In  summary,  educators  and  trainers  generally  agree  that  reflection  is  a  desirable 
activity  for  learners  to  engage  in,  but  has  proven  to  be  relatively  difficult  to  promote.  In 
this  article  we  have  described  a  specific  situation  in  which  systematic  efforts  to  promote 
reflection  achieved  only  limited  success.  A  list  of  factors  we  believe  individually  and 
collectively  affect  the  degree  to  which  reflection  occurs  was  presented  with  the 
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acknowledgment  that  there  is  scant  research  to  support  our  beliefs.  Clearly,  research  is 
needed  if  we  are  to  develop  a  proven  set  of  strategies  for  promoting  the  reflection-on- 
action  (Schon,  1987)  attempted  in  this  effort.  Kottkamp  (1990)  also  presents  ideas  on 
how  to  promote  reflection  that  are  deserving  of  additional  research. 

To  complicate  the  issue  even  further,  Liston  and  Zeichner  (1996)  have  posited  a 
five-part  taxonomy  of  reflection  of  which  reflection  immediately  (or  at  least  shortly  after) 
upon  completion  of  the  action  is  only  one  type.  According  to  Liston  and  Zeichner 
reflection  can  occur:  rapidly  during  an  action;  thoughtfully  during  an  action;  briefly  as  a 
review  after  action;  systematically  over  a  period  of  time  after  action,  or  long-term  as  one 
attempts  to  develop  formal  or  informal  theory.  As  noted,  we  have  dealt  only  with 
reflection  immediately  (or  at  least  within  a  few  days)  after  completion  of  the  learning 
activity.  Thus,  in  addition  to  the  variables  we  discussed  that  are  relevant  to  this  one  fonn 
or  reflection,  there  no  doubt  are  other  variable  to  be  explored  to  related  to  the  other  forms 
of  reflection  contained  in  Liston  and  Zeichner's  taxonomy. 

The  authors'  reflection  on  reflection  leads  us  to  two  conclusions.  First,  that 
reflection  does  indeed  result  in  enhanced  learning.  And  second  that  our  understanding  of 
how  to  tap  this  powerful  strategy  is  quite  limited.  Hopefully  this  paper  will  prompt 
additional  interest  in  the  topic  eventually  resulting  in  a  larger  body  of  research  and  theory 
that  can  be  applied  across  a  variety  of  settings.  Clearly,  advances  in  both  theory  and  in 
development  and  testing  of  practical  strategies  for  promoting  reflection  is  needed  if  we 
are  to  obtain  the  benefits  claimed  for  this  important  form  of  what  is  probably  a  uniquely 
human  behavior. 
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